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(54) Parking assistance apparatus 

(57) Parking assistance apparatus that displays on 
a screen an image shot by image picking-up means (1 0) 
nnounted on a vehicle, characterized in that said parking 
assistance apparatus superimposes a guide line that is 
parallel to the line tangent to the predicted outer locus 



of the vehicle assumed when the vehicle is backed with 
the wheel at the maximum steering angle. This allows 
the driver to drive the vehicle so that the guide line will 
not overlap the image of another vehicle thus allowing 
parking that avoids any contact. 
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Description 

BACKGROUND OF THE INVENTION 

5 1 .Field of the Invention 

[0001] The present invention relates to parking assistance apparatus for assisting parking by picking up the rear 
direction of a vehicle by using a vehicle-mounted camera and displaying the resulting Image on a screen that can be 
viewed from the driver's seat, and In particular to parking assistance apparatus for assisting parking by superimposing 
10 a guide on the screen. 

2.Descrlptlon of the Related Art 

[0002] Driving assistance apparatus has been in widespread use that provides the driver of the view at the rear 
^5 direction of a vehicle without the driving looking into a rearview mirror by mounting a vehicle-mounted camera on the 
rear trunk of a vehicle in a rear direction and presenting the image obtained from the vehicle-mounted camera to the 
driver. In particular, in recent years, driving assistance apparatus has been on the market (for example, Japanese 
Patent Laid-Open Hel 11-337740) that superimposes the predicted locus of the tires of a vehicle corresponding to the 
steering angle on the image shot by the camera and presented to the driver In order to notify the predicted direction 
^0 of the vehicle. 

[0003] However, driving assistance apparatus that superimposes the predicted travel locus moving with the variation 
in the steering angle on the image shot by a vehicle-mounted camera has outstanding problems. 
[0004] A first problem is that a steering angle sensor is expensive and a processing unit to compute the predicted 
locus of the travel direction of the tires by using sensor information from the steering angle sensor is also required thus 

25 increasing the price of the vehicle. A second problem is that errors in the predicted locus are accumulated as the 
steering angle sensor is used for longer time thus causing a displacement from the actual locus. Another problem is 
that an unskilled driver cannot easily determine the parking start position or timing to turn back the wheel when he/she 
has checked the predicted locus moving with the variation in the steering angle of the wheel. The driver needs some 
practice before he/she has accustomed to the system. 

30 [0005] Thus, as driving assistance apparatus that allows easy parking without using a steering angle sensor is pro- 
vided for example in the Japanese Patent Laid-Open Hei 1 2-078566 or Japanese Patent Lald-Open 2000-1 68475 . 
[0006] To perform end-to-end parking or side-by-side parking of a vehicle including parking of a vehicle in a garage, 
parking is often accomplished by backing the vehicle with wheel position fixed (for example rest steering to left) once 
the start position of parking is found. A related art technology described in the Japanese Patent Lald-Open Hei 

35 12-078566 focuses on this fact and discloses a technology that superimposes on the image picked up by a vehicle- 
mounted camera the vehicle space at the destination the vehicle is expected to reach in case the vehicle is backed 
with the wheel turned at the maximum steering angle. 

[0007] A related art technology described in the Japanese Patent Laid-Open 2000-168475 guides the parking pro- 
cedure by displaying the predicted vehicle travel locus assumed when the wheel is rest steered. This is based on the 
40 Idea that the driver has only to rest steer the wheel to back the vehicle once he/she has guided the vehicle so that the 
destination of the predicted locus will reach the parking space. This philosophy Is basically the same as the related art 
in the Japanese Patent Laid-Open Hei 12-78566, 

[0008] Fig. 37 Is a schematic view of a screen display example of related art driving assistance apparatus. In Fig. 
37, a rear end 1 a of the vehicle and a rectangular area guide 1 6 showing as a guide the approximate size of the vehicle 

^5 space are displayed. Assume a case where the vehicle 1 is parked Into parking space 4 before another vehicle 2 that 
Is parked, as shown in Fig. 38A, In this case, the driver watches the screen display shown In Fig. 38B and moves the 
vehicle 1 so that the position of the area guide 1b described referring to Fig. 37 will match the space adjacent to an 
image 2a of another vehicle 2. Then, backing the vehicle with the wheel fully turned to left brings the vehicle 1 into the 
space of the area guide 1b. The driver has only to position the vehicle at the start of parking procedure so that the 

50 position of the area guide on the screen matches the target location, then back the vehicle with the wheel rest steered. 
[0009] Related art driving assistance apparatus in Figs. 38A and 38B still has problems that the alignment of the 
vehicle at completion of parking is difficult and that the vehicle could come In contact with another vehicle. 
[0010] Related art apparatus described in the Japanese Patent Laid-Open Hei 12-078566/2000 assumes parking 
by backing a vehicle until the vehicle turns 90 degrees with the steering angle set to the maximum. Thus, the parking 

55 space shown in the area guide is too far from the parking start position so that the parking space (area guide) in the 
screen is very small thus making it difficult to align the vehicle at the end of parking. That is, the position of the vehicle 
at the end of parking is often too close to or too far from the adjacent vehicle. 

[001 1 ] The method to change the direction of a vehicle 90 degrees at once differs from the vehicle guidance method 
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of side-by-side parking conventionally used by many drivers, which perplexes the driver. As shown in Fig. 39, the 
related art vehicle guidance method of side-by-side parking comprises: the step of driving the vehicle 1 to bring the 
vehicle close- to the parking space 4 with the vehicle at perpendicular to the parking space 4 (step P1); the step of 
directing the rear of the vehicle toward the parking space 4 by advancing the vehicle with the wheel turned in the 

5 direction opposite to the parking space (step P2); turning back the wheel and backing the vehicle to place the vehicle 
In the center of the parking space 4 parallel to other vehicles (step P3); and backing the vehicle with the wheel held 
straight to place the vehicle in the parking space 4 (step P4). It is a problem to oblige the driver to follow a parking 
method other than this approach. Rather, it is preferable to diffuse driving assistance apparatus to assist the related 
art vehicle guidance method for side-by-slde parking. 

10 [0012] In related art apparatus described in the Japanese Patent Laid-Open 2000-1 68475, only the predicted travel 
locus assumed when the wheel is In straight position and when the wheel is rest steered. In case the parking space 
Is split by a parking border line painted on the road surface, It is possible to back the vehicle using the parking border 
line as a guide. In the absence of a parking border line, the driver must use the predicted travel locus on the screen 
as a guide to back the vehicle, In such a case, the vehicle has a higher possibility to come In contact with an adjacent 

IS vehicle. This will be explained referring to Figs. 40A and 40B. 

[0013] Fig. 40A shows the state of the step P3 for side-by-side parking. A parking border line shown by a numeral 
6 in Fig. 39 Is not displayed on the road surface. Fig. 40B shows the Image shot by a vehicle-mounted camera on the 
screen, where the predicted travel locus of the vehicle Is superimposed. Fig. 40B shows that the predicted travel locus 
shows that the predicted travel locus 6 while the wheel is rest steered grazes a point between the front tire and the 

^0 rear tire of the Image of the vehicle parked in the parking space. As seen from this image, the predicted travel locus 6 
does not overlap the adjacent vehicle image 2. So, It seems that backing the vehicle as indicated does not bring the 
vehicle in contact with the adjacent vehicle 2. 

[0014] However, checking the detailed situation, the predicted travel locus 6 crosses the straight line LT connecting 
the ground points of the front and rear tiros of the vehicle imago 2a. It is sure that backing the vehicle in this direction 
?5 brings the vehicle 1 Into contact with the adjacent vehicle 2. As such, using the predicted travel locus alone makes it 
difficult to determine the contact of the vehicle with surrounding vehicles. 

SUMMARY OF THE INVENTION 



[0015] An object of the invention is to provide parking assistance apparatus that can improve the alignment accuracy 
of the parking start position. 

[0016] Another object of the invention is to provide parking assistance apparatus that assists parking by using a 
method similar to the vehicle guidance method in a parking procedure often conventionally employed by a large number 
of drivers. 

[0017] Another object of the invention is to provide parking assistance apparatus that allows easy determination on 
the contact with surrounding obstacles. 

[0018] The aforementioned objects are attained by providing parking assistance apparatus that displays on a screen 
an image shot by image picking-up means mounted on a vehicle, characterized in that said parking assistance appa- 
ratus comprises control means for superimposing on said screen a guide parallel to a line that is tangent to the outer 
predicted travel locus of said vehicle and apart from said outer predicted travel locus at a required distance In said 
outer direction assumed when said vehicle is backed with the wheel of said vehicle operated. With this configuration, 
it is possible to know In the screen a contact with an obstacle such as an adjacent vehicle assumed when the steering 
angle is operated. This avoids any contact with an obstacle in advance. 

[0019] The objects are attained by providing parking assistance apparatus that displays on a screen the image shot 
by Image picklng-up means mounted on a vehicle, characterized in that said pari<ing assistance apparatus comprises 
control means for superimposing on said screen a guide parallel to a line that is tangent to the outer predicted travel 
locus of said vehicle and apart from said outer predicted travel locus at a required distance in said outer direction 
assumed when said vehicle is backed with the wheel of said vehicle at the maximum steering angle. With this config- 
uration, it is possible to grasp the possibility of contact with an adjacent vehicle when the wheel is turned to its limit 
even in a vehicle where a steering angle sensor is not mounted. This allows a driver who is not skilled in driving to 
move the vehicle to the parking start position referring to guide lines and perform side-by-side parking or end-to-end 
parking without undue stress. 

[0020] Preferably, said parking assistance apparatus is characterized in that said guide is a guide line. Discrete points 
may be displayed as a guide but display as lines facilitates determination on the possibility of contact. 
[0021] Preferably, said parking assistance apparatus is characterized in that the control means comprises means 
for superimposing on said screen an auxiliary guide line for assisting said guide line, said auxiliary line extending in 
the direction that crosses said guide line at a predetermined angle. With this configuration, it is easy to grasp the 
position of the vehicle from the display of the auxiliary line even in case the relation between the guide line and the 
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target object is not clear. 

[0022] Preferably, said parking assistance apparatus Is characterized in that said control means comprises means 
for superimposing on said screen a guide parallel to a line that is tangent to the outer predicted travel locus of said 
vehicle and at a required distance from said outer predicted travel focus in said outer direction assumed when said 
5 vehicle is backed with the wheel of said vehicle at the maximum steering angle in the direction opposite to said guide 
line. With this configuration, the minimum turning radius of the vehicle assumed when the wheel is turned in the opposite 
direction is known from an auxiliary line, thus making it easy to determine a contact with another object that may take 
place when the wheel is turned In the opposite direction. This makes It possible to easily grasp the timing of turning 
back the wheel. 

10 [0023] Preferably, said parking assistance apparatus is characterized in that the angle formed by said guide line and 
said auxiliary line is 90 degrees. A guide line Is parallel to a predicted locus so that an auxiliary line is perpendicular 
to the predicted locus. When the guide line becomes parallel to the vehicle, the auxiliary guide line becomes parallel 
to the rear end line of the vehicle. This clarifies the relation between the vehicle and the guide line. 
[0024] Preferably, said parking assistance apparatus is characterized in that said display means displays a mark 

15 indicating a vehicle width or a mark indicating a vehicle width having a required distance margin. Display of this mark 
allows a driver to check the width of the vehicle or width of the vehicle with a margin on the guide line thus facilitating 
determination on the relation to other objects or parking start position. 

[0025] The aforementioned objects are attained by parking assistance apparatus that displays an Image shot by 
image picklng-up means mounted on a vehicle, characterized in thai said parking assistance apparatus comprises 
20 control means for superimposing on said screen cross-shaped guide lines slanted 45 degrees right and left away from 
the center axis of said vehicle and at a predetermined distance from the rear end of said vehicle. With his configuration, 
it is made easy to detennine whether the vehicle Is slanted 45 degrees away from the target thus allowing a beginner 
to easily park a vehicle. 

[0026] Preferably, said parking assistance apparatus is characterized in that said control means forms cross-shaped 
25 guide lines with two guide lines parallel to lines that are tangent to the outer predicted travel locus of said vehicle and 
at a required distance from said outer predicted travel locus in said outer direction assumed when said vehicle is backed 
with the wheel of said vehicle at the maximum right/left steering angle. This allows the cross-shaped guidelines to be 
shared by parking in the right direction and In the left direction. The guide lines are thus simple and have wide appli- 
cations. 

30 [0027] Preferably, said parking assistance apparatus is characterized in that said control means displays intersection 
marks and marks indicating the positions away from said intersection points by the vehicle width or marks Indicating 
the positions away from said intersection points by the vehicle width having a required distance margin. 
[0028] Display of these marks allows a driver to check the width of the vehicle or width of the vehicle with a margin 
on the guide line thus facilitating determination on the relation to other objects or parking start position. 

35 [0029] Preferably, said parking assistance apparatus is characterized in that said picked up image displayed on said 
screen has undergone processing to remove lens distortion of said shooting apparatus. This displays a straight line in 
the real space as a straight line on the screen, thus allowing the driver to intuitively grasp the orientation of the vehicle 
or coincidence of a guide line with the target. 

[0030] Preferably, said parking assistance apparatus Is characterized in that said picked up image displayed on said 
40 screen has undergone viewpoint conversion processing to convert an image shot by said image picking-up means to 
an image shot by an imaginary camera installed at an arbitrary position. A camera must be installed in a limited place 
so that it is difficult to pick up an optimum image by using a vehicle-mounted camera that can be used as a reference 
to parking. The viewpoint conversion processing converts an Image to one obtained when a camera is assumed In an 
optimum place. This obtains an optimum image for parking. In this case, the viewpoint conversion processing preferably 
45 provides the same straight line in the image after viewpoint conversion as In a real space. This makes it easy to grasp 
the relationship on the screen and to determine whether the vehicle is parallel to or perpendicular to another vehicle 
or a parking border line. 

[0031] Preferably, said parking assistance apparatus is characterized in that said picked up Image has undergone 
the viewpoint conversion processing where a straight line in the real space is represented in an image after viewpoint 
conversion. A straight line in the real space is displayed as a straight line, which assures an easy-to-watch display. 
[0032] Preferably, said parking assistance apparatus is characterized in that said parking assistance apparatus dis- 
plays said guide lines in accordance with the rules of perspective. That is, a farther line is displayed in a thin line, and 
a closer line is displayed in a bold line. Guide lines are displayed as drawn on the road surface. This eliminates a sense 
of incongruity on the part of a driver. 
55 [0033] The aforementioned objects are attained by providing parking assistance apparatus characterized in that said 
parking assistance apparatus displays, rather than said guide lines, points equldistantly arranged by a predetermined 
distance on said guide lines. 

[0034] Preferably, said parking assistance apparatus Is characterized in that said guide lines are displayed by chang- 
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20 



ing the display colors of the areas split by said guide lines. This simplifies a guide on the screen thus providing an 
easy-to-watch guide. 

[0035] The aforementioned objects are attained by providing parking assistance apparatus that displays on a screen 
an image shot by image picking-up means mounted on a vehicle, characterized in that said parking assistance appa- 

5 ratus comprises control means including means for splitting said screen Into a first area for displaying an area in the 
neighborhood of a vehicle and a second area for displaying the area except said area in the close proximity of the 
vehicle, means for viewpoint-converting an Image of the neighborhood of a vehicle out of said image to an image where 
a straight line in the real space is represented in a straight line on the screen and displaying the resulting image in said 
first area, means for displaying in said second area an image of the neighborhood of a vehicle out of said image as a 

10 panoramically viewpoint-converted image continuous to the Image displayed in said first area, and means for displaying 
the boundary of said first area and said second area. 

[0036] Preferably, said parking assistance apparatus is characterized in that said means for displaying the boundary 
displays separately the range where said vehicle can pass and the range where the vehicle cannot pass . With this 
configuration, the range where the vehicle does not pass with the wheel rest steered is made clear thus facilitating 
15 deternilnation on a contact with other objects. 

[0037] Preferably, said parking assistance apparatus is characterized in that said parking assistance apparatus su- 
perimposes a guide such as the aforementioned guide on the screen split into two areas with a boundary. With this 
configuration, the parking procedure is made easier. 

[0038] The aforementioned objects are attained by providing parking assistance apparatus that displays on a screen 
an Image picked up by Image picklng-up means mounted on a vehicle, characterized in that said parking assistance 
apparatus comprises control means for superimposing on said picked up image a guide showing the range where said 
vehicle passes when said vehicle is backed. With this configuration, the range where the vehicle does not pass with 
the wheel rest steered is made clear thus facilitating determination on a contact with other objects. 
[0039] Preferably, said parking assistance apparatus is characterized in that said parking assistance apparatus su- 
25 perimposes on said screen the guide lines as well as a guide showing the range where said vehicle passes when said 
vehicle is backed. With this configuration, the parking procedure is made easier 

BRIEF DESCRIPTION OF THE DRAWINGS 

30 [0040] 

Fig. 1 Is a block diagram of parking assistance apparatus according to an embodiment of the invention. 
Fig. 2 explains a mapping table. 

Fig. 3 explains the mounting positions of two rean/iew cameras mounted on a vehicle and picked up images from 
35 the rearview cameras. 

Fig. 4 shows an image created by synthesizing picked up images from the two rearview cameras shown in Fig. 3. 
Fig. 5A shows the vehicle placed near the parking space 45 degrees slantwise away from the parking space. 
Fig. 5B shows the parking start position assumed when the vehicle is turned 90 degrees at a time with respect to 
the parking space. 

^0 Fig. 6 explains a guide line L2 according to the first embodiment of the invention. 

Fig. 7 explains an auxiliary guide line L3 according to the second embodiment of the invention. 
Fig. 8A shows the auxiliary guide line superimposed on an image of a parked vehicle. 

Fig. 8B Is a schematic view showing the relation between the vehicle and another vehicle in the screen display of 
Fig. 8A. 

^5 Fig. 9A is a schematic view showing the position of the vehicle when the vehicle is to be parked in a left-backward 

parking space. 

Fig. 9B is a schematic view showing the position of the vehicle when the vehicle is to be parked in a right-backward 
parking space. 

Fig. 10 shows guides superimposed on the monitor screen. 

Fig. 11 is a schematic view showing the relation between the position of the vehicle and the position of another 

vehicle in accordance with the side-by-side parking procedure. 

Fig. 12 shows the state of a monitor screen corresponding to each state in Fig. 11 . 

Fig, 13 is a schematic view showing the relation between the position of the vehicle and the position of another 
vehicle in accordance with the parking procedure used when the start position of side-by-side parking is too away 
55 from another vehicle . 

Fig. 14 shows the state of a monitor screen corresponding to each state in Fig. 13. 

Fig. 15 is a schematic view showing the relation between the position of the vehicle and the position of another 
vehicle in accordance with the parking procedure used when the start position of side-by-side parking is too close 
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to another vehicle. 

Fig. 16 shows the state of a monitor screen corresponding to each state in Fig. 15. 

Fig. 17 explains the end-to-end parking procedure typically followed by a large number of drivers. 

Fig. 18 Is a schematic view showing the relation between the position of the vehicle and the position of another 

vehicle in accordance with the end-to-end parking according to the first embodiment of the invention. 

Fig. 19 shows the state of a monitor screen corresponding to each state In Fig. 18. 

Fig. 20 Is a schematic view showing the relation between the position of the vehicle and the position of another 
vehicle in accordance with the parking procedure used when the start position of end-to-end parking is too away 
from another vehicle. 

Fig. 21 explains guide marks according to the second embodiment of the invention. 
Fig. 22 explains guide marks according to the second embodiment of the invention. 

Fig. 23 is a schematic view showing the side-by-slde parking procedure using guide marks according to the second 
embodiment of the Invention. 

Fig. 24 Is a schematic view showing the parking procedure used when the vehicle is too beyond the ideal position 

in sIde-by-side parking that uses guide marks according to the second embodiment of the invention. 

Fig. 25 is a schematic view showing the parking procedure used when the vehicle is too before the ideal position 

in side-by-side parking that uses guide marks according to the second embodiment of the invention. 

Fig. 26 explains guides according to the third embodiment of the Invention, 

Fig. 27 shows transition of the monitor screen assumed when side-by-side parking is performed using guides 
according to the third embodiment. 

Fig. 28 explains guide marks according to the fourth embodiment of the Invention. 
Fig. 29 explains guide marks according to the forth embodiment of the Invention. 
Fig. 30 explains guides according to the fifth embodiment of the invention. 

Fig. 31 is a schematic view showing the side-by-side parking procedure using guides according to the fifth em- 
bodiment of the invention. 

Fig. 32 explains guides according to the sixth embodiment of the Invention. 

Fig. 33 is a monitor screen transition diagram assumed when side-by-side parking procedure Is followed using 

guides according to the sixth embodiment of the Invention. 

Fig. 34 explains guides according to the seventh embodiment of the invention. 

Fig. 35 explains guides according to the eighth embodiment of the invention. 

Fig. 36 explains guides according to the ninth embodiment of the invention. 

Fig. 37 explains side-by-side parking according to the related art. 

Fig. 38A Is a schematic view showing the related art relation between the position of the vehicle and the position 
of another vehicle In accordance with the side-by-slde parking. 
Fig. 38B shows the monitor screen in the state of Fig. 38A. 

Fig. 39 explains the procedure of sIde-by-side parking typically performed by a large number of drivers. 

Fig. 40A is a schematic view showing the state where the vehicle is brought near another vehicle by applying the 

related art. 

Fig. 40B shows the monitor screen in the state of Fig. 40A. 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0041] An embodiment of the invention will be described referring to drawings, 

[0042] Fig. 1 Is a block diagram showing parking assistance apparatus according to an embodiment of the invention. 
Parking assistance apparatus according to the Invention comprises image picking-up means 10, an image processor 
20 as control means of this embodiment, and display means 30. 

[0043] The image picking-up means 10 comprises N pieces of image picking-up means mounted on a vehicle. For 
example, the image picklng-up means comprises total eight cameras: two cameras mounted on the front of the vehicle 
for monitoring the front of the vehicle; two cameras mounted on the left side of the vehicle for monitoring the left side 
of the vehicle; two cameras mounted on the right side of the vehicle for monitoring the right side of the vehicle; and 
two cameras mounted on the rear of the vehicle (rearview camera) for monitoring the rear of the vehicle. 
[0044] Each camera 8 is typically a color or monochrome digital camera having a solid picking up device such as a 
CCD or CMOS device. Each camera is composed of for example a combination of lenses, a prism and a mirror (not 
shown). Each camera may be configured so that an incident light passing through the lenses, prism and mirror will be 
transmitted to a shooting device at a distance. 

[0045] The image picking-up means 1 0 further comprises two frame memories per camera. While one frame memory 
is retrieving picked up images from a camera, the image processor 20 is reading the stored information in the other 
frame memory (picked up image at the preceding retrieve timing). 
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[0046] The Image processor 20 comprises image syntliesis means 21 for retrieving and synthesizing plci<ed up 
images from each camera 8. mapping table looking-up means 22 for performing high-speed processing of image 
synthesis, picture signal generating means 23 for converting an image generated by the image synthesis means 21 
to a picture signal, timing generating means 24 for outputting a timing signal to the image picking-up means 1 0, mapping 
table looking-up means 22, and picture signal generating means 23. 

[0047] The image synthesis means 21 inputs picked up images from each camera 8 and processes these picked up 
images. The processing Includes (1) image transfomnation processing and (2) synthesis of transformed partial images 
(including boundary processing). The processing (1 ) and (2) may be made separately or all or partial processing may 
be made in one process. The example in Fig. 1 comprises a mapping table In order to perform processing of picked 
up images in one process. 

[0048] The mapping table Is a table for perfonning high-speed conversion of position coordinates of the pixels of a 
picked up image to position coordinates of pixels of a synthesized image. Position coordinates of the pixels of a picked 
up image may correspond to position coordinates of pixels of a synthesized Image on a one-to-one basis or plurallty- 
to-one basis. Whether position coordinates of the pixels of a picked up Image may correspond to position coordinates 
of pixels of a synthesized image on a one-to-one basis or plurality-to-one basis depends on where each camera is 
mounted and what synthesized image is generated. For example, in case boundary processing is made at the boundary 
a picked up image, position coordinates of the pixels of a picked up image may not correspond to position coordinates 
of pixels of a synthesized image on a one-to-one basis. 

[0049] Fig. 2 shows an example of a mapping table. In this example, correspondence between a synthesized image 
and mapping data is Indicated based on the pixel coordinates (I, j) of a synthesized Image. The pixel coordinate (11 . 
j1) is a pixel of an image picked up by the camera 8, and is provided with data of the X coordinate and Y coordinate 
(12, 45) of picked up image, and data of necessity (1 .0) on a per coordinate basis (mapping data example 1), In the 
example of Fig. 2, pixel data of the picked up image (12. 45) having necessity "1", that is the pixel data of the picked 
up image is directly copied to the pixel coordinates (11, j1) of a synthesized image. 

[0050] Meanwhile, pixel coordinate (j2, j2) corresponds to the section where an image picked up by the camera 8 
(camera number 1 ) overlaps an image shot by the adjacent camera 8 (camera number 2). X and Y coordinate data 
and necessity data of images picked up by each camera 8 having the camera numbers 1 and 2 are provided (mapping 
data example 2). 

[0051] In this case, the pixel value of (12, j2) can be determined for example as follows: 

Pixel value ={(pixel value of coordinates of a camera of camera 
number 2 (56, 80))xo.5+(pixel value of coordinates of a camera 
of camera number 1 (10, 10))x0.3)/(0.5 + 0,3) 

[0052] By creating a table that represents this con^espondence through calculation or manual work, it is possible to 
generate at a high sped a synthesized image representing the whole ground around a vehicle and a synthesized image 
representing a partial ground around a vehicle (for example the rear ground of a vehicle or side ground of a vehicle) . 
For example, an Image to represent a wide range in one screen generated by synthesizing side by side the picked up 
images of a camera for shooting the rear left area and a camera for shooting the rear right area may be synthesized 
at a high speed by using this mapping table. 

[0053] Mapping tables may be provided in the image synthesis means 21 or external to the image synthesis means 
21. A processor (not shown) In the image synthesis means 21 generates synthesized Images at a high speed by 
referencing any of the mapping tables for generating necessary synthesized images. It goes without saying that a 
faster processor can generate synthesized images without using mapping tables. 

[0054] Specifically, a mapping table is written into a ROM (including erasable programmable ROM such as EEPROM) 
or RAM. A mapping table may be written into a RAM or ROM by the processor in the aforementioned image synthesis 
means 21 while calculating the mapping table data. Or, for example, mapping table data provided as firmware may be 
written into the RAM or ROM by using data transfer means such as a communication line and disk drive. 
[0055] The display means 30 is typically a liquid crystal display and may be another display device such as a plasma 
display. The display means 30 may be also used as a vehicle-mounted GPS terminal display (so call car navigation 
system display) or separately provided. 

[0056] Fig. 3 is a schematic view showing the vehicle-mounting position of two cameras L, R for shooting the rear 
direction of a vehicle and picked up images from each camera L, R. In this example, the two cameras L and R are 
mounted back to back across the longitudinal center line of the vehicle in order to obtain a 1 80-degree view of the rear 
of the bumper of the vehicle. Fig. 4 shows a schematic view of a synthesized image of the rear direction of a vehicle 
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from an image from each camera L and R using the aforementioned mapping table. Also, in this example of Fig. 4, 
images from two cameras are synthesized and viewpoint-converted so that the parking border line 5 which is adjacent 
to an image 2a of another vehicle 2 is displayed as a straight line. 

[0057] A vehicle-mounted camera has to shoot a wide range of images so that a wide angle camera is used. Thus 
5 the picked up image is considerably distorted. It is preferable to provide an image where distortion due to camera 
lenses is removed, when a picked up image is converted and synthesized into a real image using mapping tables. Fig. 
4 is an example of a synthesized image by two cameras. For a single wide angle camera also, a similar Image is 
obtained by removing lens distortion in order to display a parking border line as a straight line. 
[0058] As mentioned earlier, a general vehicle guidance method of side-by-side parking roughly follows four steps 
fo P1 to P4 (Fig. 39). A major characteristic of this method is to move the vehicle 1 to the parking start position of step 
P2 in Fig. 39. That is, the driver can check the parking space location directly with his/her eyes when backing the 
vehicle for parking. The parking start position is slanted away from the parking space 4 and is closer to the same space 
in order to shoot the entire parking space with vehicle-mounted cameras as well as facilitate wheel operation. Backing 
the vehicle with the wheel rest steered brings the vehicle straight into the center of the parking space 4. 
f5 [0059] Fig. 6A Is a schematic view showing the state of the vehicle position of the step P2. Fig. 5B is a schematic 
view showing the parking start position to park the vehicle while turning the vehicle by 90 degrees at once. As under- 
stood from comparison of both figures, the parking start position Is closer to the parking space 4 in Fig. 5A than In Fig. 
SB. The camera captures a wider view of the parking space 4 In Fig. 6A. In Fig. 5A, there is no possibility that most of 
the area 4a of the parking space 4 is concealed by an adjacent vehicle from a camera unlike in Fig. 58 
20 [0060] Thus, in this embodiment, a guide to accurately position the vehicle 1 in the state in Fig. 5A (step P2 in Fig. 
29) is superimposed on the screen, in order to assist general vehicle guidance In side-by-side parking. Guide line 
according to this embodiment will be detailed. 

[0061] Fig. 6 explains the first embodiment of guide lines. A straight line (segment) drawn on the road surface at 
apredetermined spacing in parallel with a tangent line L1 of the predicted travel locus 6 in the maximum steering angle 
25 js assumed as a first guide line L2 in this embodiment. Fig. 6 shows the relation between the predicted travel locus 6 
of the vehicle, Its tangent line L1 , and the first guide line L2. 

[0062] The reason why the line at a required spacing from the tangent line L1 of the predicted travel locus 6 is 
described below. A vehicle is moved so that the guide line L2 will coincide with the parking border line of the adjacent 
vehicle. Then, while the vehicle is backed with the steering angle at the maximum, said required spacing remains as 
a gap between the vehicle and the adjacent vehicle 2 in the parking space 4 where the vehicle 1 is to be parked. This 
avoids a contact with the adjacent vehicle 2. Thus, the spacing between the tangent line L1 and the guide line L2 may 
be a spacing between the vehicle to park and the adjacent vehicle. This spacing Is based on the real road surface so 
that said required spacing on the screen Is adjusted to the screen display 

[0063] In case a vehicle is to be parked, a parking border line 5 is useful. Once the vehicle 1 is aligned so that the 
55 guide line L2 will coincide with the parking border line 5 using the parking border line 5 as a target, backing the vehicle 
1 with the steering angle at the maximum parks the vehicle 1 in the center of the parking space 4. There may be a 
case where the parking border line 5 is unavailable, in such a case, a line (line LT in Fig. 40) connecting the ground 
points of the front and rear tires of the vehtele parked In space adjacent to the parking space 4 Is used as a target. The 
vehicle is guided so that the guide line L2 will coincide with the target then the vehicle is backed with the steering angle 
40 at the maximum. This safely parks the vehicle in the parking space 4. 

[0064] The target the first guide line L2 should coincide with is hereinafter referred to as Target A. 
[0065] In this embodiment, a straight line (segment) crossing the first guide line L2 on the road surface, an orthogonal 
straight line (segment) in this example is assumed as the second guide (auxiliary guide) L3. When the overlapping of 
the first guide line L2 and the target is not clear on the screen, the second guide line L3 can be used as an auxiliary 
alignment tool. This also enhances the alignment accuracy of the parking start position. Fig, 7 is a schematic diagram 
showing the predicted travel locus 6 of the vehicle, the first guide line L2 and the second guide line L3. 
[0066] By determining whether a target that forms a predetermined angle (typically perpendicular) with Target A 
(hereinafter referred to as target B) on the road surface is In a predetermined geometric relation with the guide line L3 
(typically parallel), coincidence of the guide line L2 with Target A is detennined. 
50 [0067] For example, as shown in Fig. 7, assuming that said predetennined angle is "perpendicular" and said geo- 
metric relation is "parallel," and Target B as a segment connecting the front ends of other vehicles 2 present in the 
parking space the vehicle is spaced, in case Target B Is not parallel to the guide line L3, it Is understood before starting 
parking procedure that the guide line L2 does not coincide with Target A. This prevents failure in parking. 
[0068] The guide line L3 is also used as a guide for determining whether the vehicle will contact the vehicle before 
the space where the vehicle is to be parked. For example. Fig. 8A shows a case where the guide line L3 coincides 
with the vehicles adjacent to the vehicle parking space. In this state, the vehicle is closer than Target B as shown in 
Fig. SB. Backing the vehicle backed with the wheel at the maximum steering angle, contact with the vehicle before the 
vehicle parking space cannot be avoided. By guiding the vehicle 1 so that the segment connecting the front ends of 
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the vehicles adjacent to the vehicle space 4 will not be exceeded, it is possible to determine whether the vehicle will 
come into contact with the vehicle 2 before the vehicle parking space 4. 

[0069] When parking assistance apparatus is manufactured, it is necessary to consider which point on the predicted 
tire locus (point Q on the predicted locus 6 in Fig. 6) the guide L1 for creating the guide line L2 will contact. In other 
5 words, what angle should be formed by the vehicle in the state of step P2 in Fig. 39 and the parking space is the 
question. The guide shown on the monitor screen must be simple enough as a principle In order to awid misunder- 
standing on the part of the driver. 

[0070] With respect to these considerations, it is ideal to form the guide line L2 and the guide line L3 so that the 
guide line L2 will coincide with Target A and the guide line L3 will coincide with Target B when said "angle formed by 

10 he vehicle with the parking space" is 45 degrees. Rg. 9A and B are conceptual diagrams showing the situation where 
the guide line L2 and guide line L3 coincide with Target A and Target B respectively when the vehicle and the parking 
space form an angle of 45 degrees. The reason why an angle of 45 degrees is ideal is that the same guide lines L2 
and L3 can be used for any of the case where parking is made in a left rear direction of the vehicle (Fig 9A) and the 
case where parking is periomied in a right rear direction of the vehicle (Fig. 9B), which simplifies the guide lines L2 
and L3. That is, the roles of the guide line L2 and L3 for the case where parking is performed in a left rear direction of 
the vehicle are opposite to those in the case where parking Is performed in a right rear direction of the vehicle 
[0071] Ideal guide may not be used depending on the size of a vehicle, in such a case, it is desirable that an optimum 
angle is individually obtained as said angle and separate guides are used for left-backward parking and right-backward 
parking. This embodiment will be described later refen-ing to Fig. 28. 

20 [0072] Fig. 1 0 shows an example where the guides that are based on said guide lines L2 and L3 are superimposed 
on a picked up image from the camera (actually a synthesized image with lens distortion removed). The guide lines 
L2 and L3 play opposite roles for left-backward parking and right-backward parking as mentioned eariier, the guide 
lines L2 and L3 are not discriminated from each other in Fig. 10. In the example of Fig. 10, a guide iVIl for straight- 
backward parking and a predicted travel locus 6 at the maximum steering angle arc drawn as well as the guide lines 
25 L2 and L3. " 

[0073] As in the embodiment, by superimposing the guide lines L2 and L3 on the picked up image as well as super- 
imposing a predicted travel locus at the maximum steering angle, it is made easy to determine the wheel turn back 
timing and the parking start position even when a costly steering angle sensor is not mounted on a vehicle or when 
the dnver is not skilled. It is practically sufficient to super impose only the guide lines L2 and L3 according to the 
30 . embodiment on the monitor screen. 

[0074] Next, the side-by-side parking using the guide lines L2 and L3 will be described. In the following description 
fnft«r V ' ^"'^ correspond to four steps of side-by-side parking shown in Fig. 39, unless otherwise specified' 
[0075] Figs. 11 and 12 show a case where an Ideal sIde-by-side parking without useless operation is performed 
States before and after each of the steps P1 through P4 are linked by arrows to show the parking sequence from start 
Of parking to end of parking. Fig. 11 is a schematic view showing how the relation between the vehicle 1 and its parking 
space 4 and the relation between the vehicle and the vehicles 2 adjacent to the parking space change when each step 
IS followed. Fig. 12 shows the monitor screen con-esponding to the situations before and after each step in Fig 11 
[0076] What counts most in parking using the guide lines L2 and L3 is what criteria are used to determine that step 
PI IS complete. According to the experiment environment of the inventor, an empirical conclusion is that the timing is 
when the vehicle 1 Is apart from Line C connecting the front ends of the vehicles adjacent to the vehicle parking space 
4 by approximately 50 to 60 centimeters and the driver's seat is in the center of the vehicle parking space 4 
[0077] Such criteria are very ambiguous so that it is practteally impossible for a driver to park his/her vehicle in the 
same position in all cases. The guide lines L2, L3 in accordance with the embodiment allow any dislocation of the end 
position of step PI from the ideal position to be corrected by inserting straight advancement or backing between steps 
P2 and P3 In the parking procedure. This Is detailed referring to two corresponding examples (Figs. 13 and 14, Figs. 

[0078] The first example is a case where the vehicle cannot be positioned in an Ideal location in step P1 and the 
resulting distance between the vehicle 1 and said line C is too long. In this case, as shown in Figs. 1 3 and 1 4 a straight 
backward step P2a is provided between steps P2 and P3 to correct the vehicle position. In this case, the point for 
completing parking are (i) and (ii) detailed below. 

(I) Detennination on end of step P2 

[0079] End of step P2 is determined by whether the vehicle 1 is slanted 45 degrees away from the parking space 4 
As shown in the third state in Fig. 13, step P2 is assumed to be complete when the vehicle is guided so that the guide 
line L2 is parallel to target A (a parking border line splitting the vehicle parking space and the adjacent parking space 
If any; othenwise a line connecting the ground points of the front and rear tires of the adjacent vehicle, and so on) and 
the guide line L3 is parallel to Target B (a line connecting the front ends of the adjacent vehicles in case the parking 
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border line is absent; othenwise a line indicating the front end of the parking space, and so on; Line C is Target B in 
tlie example of Fig. 13). as shown in the third state of Fig. 13. 

(it) Deternnination on start of step P3 

5 

[0080] Start position of step P3 must be such that the vehicle is placed approximately in the center of the parking 
space 4 when the vehicle is backed with the wheel at the maximum steering angle to bring the vehicle 1 parallel to the 
orientation of the parking space 4. That is, the guide line L2 must coincide with Target A. Thus, as shown in the fourth 
state of Fig. 13, the driver returns the wheel straight and backs the vehicle once he/she has detennined that step P2 
10 is complete. Again, as shown in the fourth state of Fig. 13, the driver turns the wheel at the maximum steering angle 
when the guide line L2 has coincided with a line connecting the ground points of the front and rear tires of the adjacent 
vehicle. 

[0081] The two steps (i) and (ii) are equivalent to step P2 in the ideal side-by-side parking in Fig. 39. Then, executing 
steps P3 and P4 completes parking. 
15 [0082] The second example is a case where the vehicle cannot be positioned in an ideal location in step P1 and the 
position of the vehicle is far before the ideal position (close to the adjacent vehicle before the parking space at the end 
of step PI . In this case, as shown in Figs. 1 5 and 1 6, a straight fonward step P2b is provided between steps P2 and 
P3 to correct the vehicle position. In this case also, the point for completing parking are (iii) and (iv) detailed below. 

20 (ill) Determination of end of step P2 

[0083] End of step P2 is determined by whether the vehicle 1 is slanted 45 degrees away from the parking space 4, 
As shown in the third state in Fig. 15, step P2 is assumed to be complete when the vehicle is guided so that the guide 
line L2 is parallel to target A and the guide line L3 Is parallel to Target B (In this case Line C), as shown in the third 
25 state of Fig. 15. 

(iv) Detemnl nation of start of step P3 

[0084] Start position of step P3 must be such that the vehicle is placed approximately in the center of the parking 
30 space 4 when the vehicle is backed with the wheel at the maximum steering angle to bring the vehicle 1 parallel to the 
orientation of the parking space 4. That is, the guide line L2 must coincide with Target A. Thus, the driver returns the 
wheel straight and advances the vehicle once he/she has determined that step P2 is complete (the third state in Fig. 
1 5), to cause the guide line L2 to coincide with target A (the fourth state in Fig. 1 5). In this practice, it is necessary to 
take utmost care to check for any obstacle in the front direction of the vehicle 1 (advancing direction). 
35 [0085] The two steps (iii) and (iv) are equivalent to step P2 in the ideal side-by-side parking in Fig. 39. Then, executing 
steps P3 and P4 completes parking. 

[0086] Next, the end-to-end parking using the guide lines L2 and L3 will be described. In the end-to-end parking 
also, the guide lines L2 and L3 are used for parking. In practice, a different target that a guide will be aligned with is used . 
[0087] Fig. 1 7 explains the vehicle guidance method conventionally employed by a large number of drivers. For end- 
40 to-end parking, the driver guides the vehicle 1 position the vehicle side by side with another vehicle parked before the 
parking space 4 (step J1 ), then the driver rest steers the wheel to left to bring the rear of the vehicle close to a curb 7 
(step J2), then finally rest steers the wheel to right and backs the vehicle to bring the vehicle 1 in parallel with the curb 
7 (step J3), 

[0088] In the end-to-end parking, a reference line such as the curb 7 on the seat next to the driver at completion of 
45 end-to-end parking Is used as a target for the guide line L3. In the following description, steps J1 , J2, and J3 correspond 
to three steps of end-to-end parking shown in Fig. 17, unless otherwise specified, 

[0089] Fig. 18 shows a case where an ideal side-by-side parking without useless operation is performed. Stales 
before and after each of the steps J1 through J3 are linked by arrows to show the parking sequence from start of 
parking to end of parking. Fig. 1 8 is a schematic view showing how the relation between the vehicle 1 and its parking 
so space 4 and the relation between the vehicle and the vehicles 2 adjacent to the parking space change when each step 
is followed. Fig. 19 shows the monitor screen corresponding to the situations before and after each step in Fig. 1 8. 
[0090] What counts most in parking using the guide line L2 is what criteria are used to determine that step J1 , J2 or 
J3 is complete. The method for determining end of each step will be described. 

55 (a) Determination on end of step J1 

[0091] For the end of step J1 , according to the experiment environment of the inventor, as shown In the second state 
in Fig. 18, an empirical conclusion is that the timing is when the vehicle 1 is apart from another vehicle 2 before the 
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parking space 4 by approximately 60 to 60 centimeters and the rear end of the vehicle 1 Is behind the rear end of the 
vehicle 2 before the parking space 4 by approximately 50 to 60 centimeters. 

(b) Determination on end of step J2 

5 

[0092] End of step J2 is determined by whether the guide line L3 matches the curb 7 . In case an ideal end-to-end 
parking is perfomied, the guide line L3 coincides with the curb 7 when they have matched each other. The driver follows 
step J3 when such matching takes place, then backs the vehicle with the wheel rest steered to right to bring the vehicle 
1 in parallel with the curb 7. This complete end-to-end parking. 
10 [0093] The problem is that the criteria for completion of step J1 are very ambiguous. It is practically impossible for 
a driver to park his/her vehicle in the same position in all cases (second state in Fig. 18). Thus, in case the end position 
of step J1 is not ideal, the dislocation must be corrected by the subsequent steps. 

[0094] The guide lines L2, L3 in accordance with the embodiment allow any dislocation of the end position of step 
J1 from the ideal position to be corrected thus ensuring the parking procedure. Its specific example will be described 
15 using Fig, 20. Fig. 20, same as Fig. 18, shows the end-to-end parking. Fig. 20 differs from Fig, 18 in that step J2a is 
added after step J2 in order to correct said dislocation. 

[0095] In this example, the spacing between the vehicle 1 at the end of step J1 and another vehicle 2 before the 
parking space 4 is more than the ideal spacing (approximately 50 to 60 centimeters) . The driver cannot guide the 
vehicle 1 so satisfy the criteria for completion of said step .J2 even In case he/she has rest steered the wheel at the 
20 end of step J1 . 

[0096] In this case, that is, in case the guide line L3 and the curb 7 are parallel but do not coincide with each other 
while the driver is backing the vehicle with the wheel rest steered, the driver aborts backing with the wheel rest steered. 
The driver turns back the wheel straight to back the vehicle straight until the guide line L3 coincides with the curb 7 
(fourth state in Fig. 20) in next step J2a, then proceeds to next step J3. 
25 [0097] Additional step J2a is equivalent to step J2 in the ideal end-to-end parking in Fig. 17. Then, executing step 
J3 completes end-to-end parking. 

[0098] It has been proved that the guide line L2 and L3 explained in Fig. 10 are effective in the side-by-side parking 
and end-to-end parking according to the aforementioned embodiments. Further, displaying simple guide marks pro- 
vides the driver with a more effective material for detennination. An embodiment concerning these guide marks will be 
30 described. 

[0099] Fig. 21 explains a guide according to the second embodiment of the invention. In this embodiment, guide 
marks Q2, Q4 are attached to guide lines L2, L3. In Fig. 21 , the guide mark Q2 indicates the intersection of the guide 
line L3 and the guide line L2. The guide mark 4 is the intersection of the straight line L5 apart from the same prede- 
termined spacing as that between the tangent line L1 and the guide line L2 from the parallel line L4 apart from the 
35 tangent line L1 by the distance of vehicle width, and the guide line L3. 

[0100] In this way, the guide mark is placed at a predetemnined spacing from the line L4 tangent to the vehicle, same 
as the guide line L2 is placed apart from the tangent line L1 , because said predetermined spacing is reserved from an 
obstacle such as a vehicle on the left so that a contact may be avoided without fail. 

[0101] In particular, as shown in Fig. 22, in case a point of contact Q of the predicted backing locus is on the guide 
40 line L3, the intersection Q3 of the line L4 and the guide line L3 suggests the interior of the predicted travel locus. Thus, 
the guide mark Q4 is drawn with a predetermined spacing margin inside the predicted travel locus. In other words, the 
segment Q2-Q4 displays a range that has a margin of a predetermined spacing on the right and left over the section 
(vehicle width range) the predicted travel section (section of segment Q-Q3) overlaps the guide line L3. 
[0102] While details will be given later, when the driver parks a vehicle by using the guide mark Q2 and the guide 
^5 mark Q4 as a guide, he/she can park the vehicle just in the center of the parking space by steering the vehicle so that 
both guide marks Q2, Q4 between the right and left parking border lines when such lines are painted In the parking 
location, or between the right and left parked vehicles when such vehicles are present although the border lines are 
not painted. 

[0103] The guide lines L2, L3 and the guide marks Q2, Q4 may be displayed so that the driver can view such guide 
50 lines or guide marks. While the guide lines Q2, Q4 are displayed as marks larger than the line width of the guide lines 
L2, L3, they need not be displayed as visible marks but replaced by section of segment Q2-Q4 in a different color, or 
only the segment Q2-Q4 may be hidden so that the guide marks Q2, Q4 can be identified by the driver. 
[0104] Fig. 23 explains the side-by-slde parking using the guide marks Q2, Q4. This embodiment Is the same as the 
aforementioned side-by-side parking procedure in Fig. 11 (step PI , P2, P3, P4) but the determination on the start of 
55 step P3 is more secure in this embodiment. 

[0105] In step P3, the driver starts backing the vehicle with the wheel at the maximum steering angle and completes 
step P3 when the vehicle 1 is in a straight position in the parking space 4. In case the driver fails to properly start step 
P3, he/she cannot park the vehicle 1 in the center of the parking space, but in a position off the center either rightward 
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or leftward. To park a vehicle in the center of the parking space 4 by using the guide lines L2, L3 alone, some degree 
of skill is required. 

[0106] It is possible to accurately determine the start position of step P3. by moving the vehicle 1 Into the position 
where the guide lines Q2, Q4 are arranged uniformly between the adjacent vehicles (or between the parking border 
5 lines) when the guide lines Q2, Q4 of this embodiment are displayed and the guide line L3 is brought in parallel with 
Target B (line connecting the front ends of adjacent vehicles). 

[0107] Fig. 24 is an explanatory drawing corresponding to the side-by-side parking described referring to Fig. 13. In 
case the vehicle has passed through the ideal position of step Pi, the driver directly backs the vehicle In step P2a. 
While the guide lines L2 and L3 are used as a guide to determine the start position of step P3 in the embodiment of 
10 Fig. 13, guide marks Q2, Q4 can be referenced on top in this embodiment, thus allowing a higher-accuracy determi- 
nation of the start position of step P3. This determination Is made as mentioned in (v) below. 

(V) Detennination on start of step P3 

15 [0108] Start position of step P3 must be such that the vehicle 1 is placed approximately in the center of the parking 
space 4 when the vehicle is backed with the wheel at the maximum steering angle to bring the vehicle 1 parallel to the 
orientation of the parking space 4. In the aforementioned detemriinatlon on start of step P3 (iv), the guide is coincidence 
of the guide line L2 with Target A. On the other hand, the guide marks Q2, Q4 must be uniformly arranged by providing 
balanced distances from the adjacent vehicles. Thus, the driver turns back the wheel straight once he/she has deter- 

20 mined completion of step P2, and backs the vehicle straight The driver stops straight backing when the vehicle has 
entered the fourth state where the guide marks Q2, Q4 are uniformly arranged side by side in the parking space. The 
driver follows step P3 to start backing the vehicle with the wheel rest steered. 

[0109] This step is equivalent to step P2 in the ideal side-by-side parking in Fig. 39, thus parking the vehicle in the 
center of the parking space. 

25 [0110] Fig. 25 shows a case where the end position of step P2 is before the ideal position, which corresponds to the 
aforementioned embodiment In Fig. 1 5, on the contrary to Fig. 24. In this embodiment, the driver after the third state 
in Fig. 24, advances the vehicle straight in step P2b. In step P2b, the driver detennlnes the start of step P2b same as 
the step P3 start detennination of (v) and stops advancing the vehicle when the vehicle has entered the fourth state 
where the guide marks Q2, Q4 are uniformly arranged with respect to right/left spacing in the parking space 4. The 

30 driver then follows step P3 to park the vehicle in the center of the parking space. 

[0111] Fig. 26 explains the guides according to the third embodiment of the invention. Guides used in Fig. 26 are a 
guide mark Q5 as well as the guide lines L2, L3 and the guide marks Q2, Q4 used in the first and second embodiments. 
The guide mari< Q5 on the guide line L2 has a width with a predetermined spacing both rightward and leftward compared 
with the vehicle width, in the section Q2-Q5. The guide mark Q5 corresponds to Q4 on the guide line L3 and used 

35 when the vehicle is to be parked in the parking space in a rear left direction. The use method is the same as that for 
the guide mark Q4. 

[0112] In the example of Fig. 26, the guide lines L2, L3 near the guide mark Q2 are hidden In order to provide a 
better view of the guide mark Q2. Guide lines L2, L3 are hidden in the guide mark sections Q2-Q4 and Q2-Q5. Three 
guide lines Q2, Q4, Q5 are displayed in different colors to assure easy Identification. 

40 [0113] Fig. 27 shows an example of transition of the actual monitor screen assumed when side-by-side parking is 
perfonned using guides according to the third embodiment. The monitor example of side-by-side parking is the same 
as the procedure for side-by-side parking in Fig. 24 (procedure used when the vehicle is far from the ideal position) 
[0114] In the third state of the monitor screen in Fig. 27, the vehicle is dislocated from the ideal position. This is 
because the guide mark Q4 is not on the product of the parking border line image 5a. Thus, step P2a is executed, that 

45 is, the driver backs the vehicle straight to place the vehicle In the fourth state of the monitor screen, that Is, so that the 
guide mark Q4 will be on the extension of the parking border line image 5a. 

[0115] In the example of Fig. 27, the spacing between the guide marks Q2 and Q4 is the same as the spacing 
between the adjacent pari<ing border lines. In the fourth state, the guide mark Q2 is also on the extension of the parking 
border line image ahead. In case the spacing between the guide marks Q2 and Q4 is narrower than the spacing 
between the adjacent parking border lines, the driver backs the vehicle so that the guide marks Q2, Q4 are arranged 
equidistantly within the corresponding parking border line images. By positioning the vehicle in an ideal location (fourth 
state) then executing steps P3 and P4, the driver can parks the vehicle in the center of the parking space. 
[0116] While the guide lines 12 and L3 are slanted 45 degrees leftward and rightward respectively away from the 
center line of the vehicle, it is expected that the ideal guide lines L2, L3 cannot be used depending on the size and 
shape of a vehicle. In this case, it is desirable to individually obtain optimum angles as said angles and provide separate 
guides for rightward/leftward parking. Fig. 28 shows guides according to the fourth embodiment where separate guides 
for rightward/leftward parking are provided. 

[0117] In the fourth embodiment of the invention shown in Fig. 28 . the guide F for left-backward parking and the 
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guide G for right-backward parking are separately provided. The guide F is composed of guide lines FL2, FL3 corre- 
sponding to the guide lines L2, L3 and guide marks FQ2, FQ4 corresponding to the guide marks Q2, Q4. The guide 
G is composed of guide lines GL2, GL3 corresponding to the guide lines L2, L3 and guide marks GQ2, GQ5 corre- 
sponding to the guide marks Q2, Q5. 
5 [0118] In case the guides F, G are used, the screen is complicated when both guides F and G are displayed at all 
times. It is detected whether the wheel is being turned right or left and the corresponding guide us displayed together 
with the guide M1 as shown in Fig. 29. 

[0119] Fig. 30 explains the guides according to the fifth embodiment of the Invention. The guides used in Fig. 30 are 
a guide line C1 at an appropriate distance from the rear end of the vehicle as well as the guide lines L2, L3, Ml and 
10 the guide marks Q2, Q4, Q5 used in the third embodiment. The guide line CI is a line over which a vehicle may move 
when the vehicle is backed. This means that it is possible to guide a vehicle into an area where the guide line C1 is 
superimposed by operating the wheel. In case the vehicle is backed with the wheel at the maximum right/left steering 
angle, the vehicle moves just at the end of the guide line CI . 

[0120] When parking a vehicle using the guide line CI , the driver can roughly guide the vehicle by using the guide 
line CI as a guide in the initial stage of the parking procedure. Then the driver can guide the vehicle Into the parking 
position by using other guide lines L2, L3 and guide marks Q2, Q4, Q5. 

[0121] While straight lines in the real space are represented in straight lines like parking border lines through view- 
point-conversion of an image shot by a wide angle camera, displaying the image over the entire monitor screen results 
in distorted end portions on the screen. In this example, the image is not displayed in the upper corners 40, 41 . 
20 [0122] Fig. 31 explains the side-by-side parking using the guide line CI shown in Fig. 30. Fig. 31 corresponds to 
Fig. 24. In Fig. 31 , the guide mark Q2 and the guide mark Q4 are connected by the guide line L3. 
[0123] When the criteria for determining completion of step P1 in the side-by-side parking, the driver advances the 
vehicle straight (step P2a) after step P2 or backs the vehicle straight to correct the vehicle position to the optimum 
position. In case the guide line CI according to this embodiment is not displayed In correcting the vehicle position, a 
wide space is required for execution of step P2a. In this embodiment, the guide line CI is displayed so that it is possible 
to properly determine completion of step PI (start of step P2) as mentioned later This saves the space required for 
correction of vehicle position. 

[0124] This is because it is not necessary to constantly place the vehicle 45 degrees slanted away from the parking 
space at the completion of step PI . This is because it is made easy to detemilne whether the vehicle will come into 
30 contact with the adjacent vehicle by refemng to the guide line CI . The space required for executing each step refers 
to the space ahead of the parking space. 

[0125] In step P1 of the embodiment, the vehicle is halted slantwise away from the parking space while the vehicle 
has passed through the front of the parking space 4 and front end of the vehicle heading In the direction opposite to 
the parking space 4. The vehicle need not necessarily parked slantwise but may be parked at a right angle to the 

35 parking space. In this case, the space required for parking is wider than when parked slantwise. 

[0126] In case the end of the guide line CI facing the parking space is considerably dislocated toward the parking 
space 4 from the vehicle 3 adjacent to the parking space 4 and closer to the vehicle, backing the vehicle with the wheel 
at the maximum steering angle will bring the vehicle to the position considerably biased toward the parking space from 
the adjacent vehicle 3, that is, the vehicle may collide with the adjacent vehicle 2 beyond the parking space. In this 

40 case, the driver slightly advances the vehicle toward the adjacent vehicle 3 before the parking space to correct the 
vehicle position. 

[0127] On the other hand, in case the end of the guide line facing the parking space 4 is considerably dislocated 
toward the adjacent vehicle 3 that is adjacent to the pari<lng space 4 and closerto the vehicle, backing the vehicle with 
the wheel at the maximum steering angle will collide with the adjacent vehicle 3. In this case, the driver backs the 
vehicle toward the adjacent vehicle 2 before the parking space to correct the vehicle position. 
[0128] In any case, the driver makes adjustments so that the end of the guide line CI facing the parking space 4 will 
not come in contact with the side of the adjacent vehicle 3 facing the parking space 4 that is adjacent to the parking 
space 4 and closer to the vehicle. In this way, by using the guide line CI , it is possible to roughly guide the vehicle to 
the start position of step P2 as the initial stage of parking. 
50 [0129] In step P2, the driver backs the vehicle 1 with the wheel at the maximum steering angle and aligns the guide 
line L2 in parallel with the parking space 4 or left side of the adjacent vehicle 2 as well as aligns the guide line L3 in 
parallel with the parking space 3 or front end of the adjacent vehicle 2 or adjacent vehicle 3. Aftenvards. the driver 
follows the same procedure as the first embodiment to complete parking. 

[0130] The initial position of a vehicle being par1<ed is closely related to the extent of the travel area required in 
parking procedure. In case a sufficiently ample area cannot be provided ahead of the parking position, the area required 
for parking may not be secured depending on the parking start position, thus requiring repeated attempts. By using 
the guide line CI of the embodiment, it is possible to roughly guide the vehicle at the initial stage of parking and the 
extent of the travel area required for parking is no more dependent on the vehicle being parked. Even in case a suffi- 
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ciently ample area cannot be provided ahead of the parking position, it is made easy to park a vehicle by using the 
guide line C1 of this embodiment. Note that the guide line C1 is preferably drawn In a location corresponding to the 
position three to four meters away from the rear of the vehicle In the real space. 

[0131] Fig. 32 explains guides according to the sixth embodiment of the invention. The guides of this embodiments 
5 are basically the same as those of the fifth embodiment, with the guide line CI somewhat devised. In this embodiment, 

the monitor screen is split into the nearby area A1 at the rear of the vehicle and the remaining area (distant area) A2. 

The area A1 and the area A2 are split by a parabola-shaped boundaries (C1 , C2) . The area A1 shows an image that 

has undergone viewpoint conversion to provide a straight line In the real space in straight representation on the screen. 

The area A2 shows an Image that has undergone viewpoint conversion to provide a panoramic view of the rear of the 
^0 vehicle. Both images are continuous at the boundaries to fon-n a single image. The guide line same as that of the fifth 

embodiment is drawn on the boundaries. The remaining boundary C2 is drawn in a different color from that of the guide 

line CI. 

[0132] Displaying two differently converted images on a single screen so that the Images may be continuous on the 
boundaries results In a straight line extending across two areas A1 , A2 being represented as a polygonal line changing 
^5 its direction at a boundary. In case the guide line C1 or boundary C2 is not specified on the screen, the driver feels a 
sense of Incongruity because the parking border line that Is assumed as a straight line is represented in a polygonal 
line on the screen. 

[0133] However, when the boundaries CI , C2 for two differently converted images are explicitly shown on the screen, 
these boundaries C1 , C2 act as symbols to split the two screen areas A1 , A2 so that the driver can intuitively recognize 
20 the screen area A1 and the screen area A2 as separate areas, not a single continuous area. Thus, a polygonal line 
changing its direction at a boundary does not perplex the driver. In this way, the driver recognizes two differently con- 
verted screen areas as separate areas when the two areas are displayed to continue at a boundary, by specifying the 
boundaries. This is advantageous in that the driver does not feel a sense of incongruity when the images are not 
continuous at boundaries. 

25 [01 34] An image displayed In the image area A1 is an image in the close proximity of the vehicle that has undergone 
viewj30int conversion so that a straight line In the real space will remain straight even after viewpoint conversion. On 
the contrary, an image displayed in the image area A2 Is an image that has undergone panoramic viewpoint conversion 
so that an image outside the proximity of the vehicle will look natural. 

[0135] As shown in Fig. 26, in case an image in the close proximity of the vehicle is available as well as an image 
30 in the image area A2, parking is made possible. Displaying an image produced by the processing to convert a straight 
line in the real space to a straight line on the screen renders the guide lines L2, L3 straight lines on the screen and the 
parking border lines on the road surface indicating the parking space as straight lines. This facilitates alignment of the 
guide lines used In parking with the parking border lines on the road surface. 

[0136] In the conversion processing to convert a straight line in the real space to a straight line on the screen, the 
35 wider the image angle is, the greater the distortion of display of a three-dimensional object at the ends of the screen. 
Thus, displaying on the screen the entire image acquired using a camera capable of shooting a range wider than the 
ranged displayed in Fig. 26 provides a more distorted view as the image approaches an end of the screen. In order to 
reduce a sense of Incongruity caused by such distortion, it is preferable to cut out an image in the close proximity of 
the vehicle from the entire image and fill the monitor screen with the resulting image. 
40 [0137] In this way, the range displayed by applying the processing to convert a straight line in the real space to a 
straight line on the screen is limited to the close proximity of a vehicle. In parking procedure, it Is necessary to grasp 
the situation near the vehicle as well as a wide range to help check safety of the surroundings and understand the 
general position of the vehicle, in order to assure safe and comfortable parking. 

[01 38] This embodiment thus displays a panoramically converted image in the screen area A2 in order to help grasp 
45 the situation of a wide range. This allows the driver to grasp the situation of a wide range in the screen area A2 that 
employs panoramic conversion as well as grasp the situation of the close proximity of the vehicle in the screen area 
A1 that employs the processing to convert a straight line in the real space to a straight line on the screen, thus assuring 
safe and easy parking. 

[0139] As mentioned earlier, in a screen configuration where the boundaries CI , C2 are explicitly shown and the 
50 screen areas A1 , A2 are specifically separated, one boundary where a vehicle may pass through when backed Is 
represented as the guide line CI and the other boundary where a vehicle will never pass through when backed Is 
represented as the guide line C2. These boundaries are specified, such as represented in different colors for clear 
discrimination. Displaying the guide line C1 using the boundaries of the screen areas A1 and A2 eliminates the need 
for displaying new guide lines on the screen thus providing an easy-to-watch parking guide. 
55 [0140] Fig. 33 is a transition diagram of the monitor screen displayed when side-by-side parking is performed using 
the guides shown in Fig. 32, Determination on the initial position of start of parking is the same as that in Fig. 31 , so 
that the corresponding description is omitted. In this embodiment also, the driver can easily park the vehicle even when 
the space ahead of the parking space is not sufficient. Further in this embodiment, the driver can check safety in the 
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close proximity of the vehicle as well as check safety at a distance while watching the image in the area A2. This 
assures safe parking. 

[0141] Fig. 34 explains guides according to the seventh embodiment of the invention. This embodiment displays a 
plurality of points R on the straight lines L2, L3 as well as the guide mark Q2. The points R are arranged in predetermined 

5 spacings (for example 1 meter) from the guide mark Q2. In this embodiment, the straight lines L2 and L3 are hidden 
and the predicted locus 6 of the maximum steering angle to be displayed is hidden. The relation between the locus 6 
and the straight lines L2, L3 is the same as that In the first and second embodiments explained referring to Figs. 6 and 7. 
[0142] In this way, displaying only the guide mark Q2 and points R eliminates an easy-to-watch display without 
complicated information. A driver skilled in parking may desire correct decision on the distance by using the points R 

10 rather than complicated screen display. Thus, guides of this embodiment and guides of the sixth embodiment may be 
provided for example to let the driver choose to use which guides. 

[0143] Fig. 35 explains guides according to the eight embodiment of the invention. This embodiment is a variation 
of the seventh embodiment and also displays the guide mark Q2 and the plurality of points R. This embodiment is 
different in that the segments L2, L3 connecting adjacent points R are discretely displayed. This embodiment has the 

15 same advantage as the seventh embodiment. 

[0144] Fig. 36 explains guides according to the ninth embodiment of the invention. This embodiment displays the 
areas B1 and B2 shown in Fig. 36 out of the areas split by the straight lines L2, L3 In a color different from that of the 
other areas. This renders the boundaries of the areas straight lines L2, L3 and provides a simple display. 
[0145] While detailed display method for the guide lines CI , C2, Ml , L2, L3. FL2, FL3, GL2 is not described in Ihe 

20 aforementioned embodiments, displaying the guide line M1 for example in the same line width causes the guide line 
M1 to be floated over the road surface image, which gives the driver a sense of incongruity. To eliminate this sense of 
incongruity, the rules of perspective are preferably used to display the guide lines. That is, a closer line is displayed in 
a bold tine and a farther line Is displayed in a thin line. This allows the driver to see the guide lines CI , C2, Ml , L2, L3, 
FL2, FL3, GL2 at the same height as the road surface. 

25 [0146] In case the aforementioned parking assistance apparatus is applied to an operating vehicle, creating guide 
data to suit the target vehicle in advance and storing the data into a storage allows the guide lines CI , C2, Ml, L2, L3 
and marks Q2, Q4, Q5 to be superimposed on a camera-picked up image. In case the apparatus is applied to vehicles 
of different shapes or sizes, optimum guide data to support individual vehicles must be created and stored into storage. 
Creation of the guide data is preferably made by service personnel on a paid or free basis when a vehicle is delivered 

30 in order to create guide data optimally tailored to the user's garage. 

[0147] While the guide lines L2, L3 is assumed as data displayed as straight lines (segments) displayed on the 
monitor screen since the camera-picked up image is converted to an Image free from lens distortion (parking border 
lines as displayed as straight lines) and displayed on the monitor screen in the aforementioned embodiments, the guide 
lines L2, L3 may be displayed on the monitor screen while distorted similarly to an image shot by a wide angle camera, 

3S if wishing to display a distorted image picked-up by a wide angle camera. 

[0148] While the guide line L2 is parallel to a tangent line to the predicted travel locus at the maximum steering angle 
in the aforementioned embodiments, the predicted travel locus may vary on the screen in accordance with the variation 
in the steering angle. In this case, the guide line L2 is drawn In parallel with and outwardly from the tangent line to the 
predicted travel locus in order to move the guide line on the screen in response to a variation in the steering angle, 

40 thereby helping determine the contact with adjacent vehicles depending on the steering angle. 

[0149] While the invention has been described in terms of the specific embodiments for easy understanding of the 
inventive technology, the invention is not limited to the illustrative embodiments but may be implemented with specific 
configuration changed within the scope of the claims. 

[01 50] The invention displays guides that can measure the required outer distance along the predicted backing locus 
4s of a vehicle. Thus it Is possible to avoid possible contact with other vehicles In advance. It Is also possible for a beginner 
to perform parking perform parking in accordance with the side-by-side parking and end-to-end parking procedures 
that have been employed. 



50 Claims 

1 . Parking assistance apparatus that displays on a screen an image picked up by image picking-up means mounted 
on a vehicle, characterized in that said parking assistance apparatus comprises control means for superimposing 
on said screen a guide parallel to a line that is tangent to the outer predicted travel locus of said vehicle and apart 

55 from said outer predicted travel locus at a required distance in said outer direction assumed when said vehicle is 

backed with the wheel of said vehicle operated. 

2. Parking assistance apparatus that displays on a screen the image picked up by image picking-up means mounted 
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on a vehicle, characterized in that said parking assistance apparatus comprises control means for superimposing 
on said screen a guide parallel to a line that is tangent to the outer predicted travel locus of said vehicle and apart 
from said outer predicted travel locus at a required distance In said outer direction assumed when said vehicle Is 
backed with the wheel of said vehicle at the maximum steering angle. 

3. Parking assistance apparatus according to claim 1 , characterized in that said guide Is a guide line. 

4. Parking assistance apparatus according to claim 3, characterized In that the control means comprises means 
for superimposing on said screen an auxiliary guide line for assisting said guide line, said auxiliary line extending 
In the direction that crosses said guide line at a predetermined angle. 

5. Parking assistance apparatus according to claim 3, characterized In that said control means comprises means 
for superimposing on said screen a guide parallel to a line that is tangent to the outer predicted travel locus of said 
vehicle and at a required distance from said outer predicted travel locus in said outer direction assumed when said 
vehicle is backed with the wheel of said vehicle at the maximum steering angle In the direction opposite to said 
guide line. 



6. Parking assistance apparatus according to claim 6, characterized In that the angle fonned by said guide line and 
said auxiliary line Is 90 degrees. 

20 

7. Parking assistance apparatus according to claim 4, characterized in that said display means displays a mark 
indicating a vehicle width or a mark indicating a vehicle width having a required distance margin. 

8. Parking assistance apparatus according to claim 5, characterized In that said display means displays a mark 
25 indicating a vehicle width or a mark indicating a vehicle width having a required distance margin. 

9. Parking assistance apparatus according to claim 2, characterized in that said guide is a guide line. 

10. Parking assistance apparatus according to claim 9, characterized in that the control means comprises means 
30 for superimposing on said screen an auxiliary guide line for assisting said guide line, said auxiliary line extending 

In the direction that crosses said guide line at a predetemriined angle. 

11. Parking assistance apparatus according to claim 9, characterized in that said control means comprises means 
for superimposing on said screen a guide parallel to a line that is tangent to the outer predicted travel locus of said 

35 vehicle and at a required distance from said outer predicted travel locus in said outer direction assumed when said 

vehicle Is backed with the wheel of said vehicle at the maximum steering angle In the direction opposite to said 
guide line. 

12. Parking assistance apparatus according to claim 11 , characterized in that the angle formed by said guide line 
40 and said auxiliary line is 90 degrees. 

13. Parking assistance apparatus according to claim 10, characterized in that said display means displays a mark 
indicating a vehicle width or a mark indicating a vehicle width having a required distance margin. 

« 14. Parking assistance apparatus according to claim 11 , characterized In that said display means displays a mark 
Indicating a vehicle width or a mark Indicating a vehicle width having a required distance margin. 

15. Parking assistance apparatus that displays an image shot by image picking-up means mounted on a vehicle, 
characterized In that said parking assistance apparatus comprises control means for superimposing on said 

50 screen cross-shaped guide lines slanted 45 degrees right and left away from the center axis of said vehicle and 

at a predetemilned distance from the rear end of said vehicle. 

1 6. Parking assistance apparatus according to claim 1 5, characterized in that said control means fonns cross-shaped 
guide lines with two guide lines parallel to lines that are tangent to the outer predicted travel locus of said vehicle 

55 and at a required distance from said outer predicted travel locus in said outer direction assumed when said vehicle 

Is backed with the wheel of said vehicle at the maximum right/left steering angle. 

1 7. Parking assistance apparatus according to claim 1 5, characterized in that said control means displays i ntersection 
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marks and marks Indicating the positions away from said intersection points by the vehicle width or marks indicating 
the positions away from said Intersection points by the vehicle width having a required distance margin. 

18. Parking assistance apparatus according to claim 1 , characterized in that said picked up Image displayed on said 
screen has undergone processing to remove lens distortion of said shooting apparatus. 

19. Parking assistance apparatus according to claim 2, characterized In that said picked up image displayed on said 
screen has undergone processing to remove lens distortion of said shooting apparatus. 

20. Parking assistance apparatus according to claim 15, characterized in that said picked up image displayed on 
said screen has undergone processing to remove lens distortion of said shooting apparatus. 

21 . Parking assistance apparatus according to claim 1 , characterized In that said picked up image displayed on said 
screen has undergone viewpoint conversion processing to convert an image picked up by said image picking-up 
means to an image picked up by an imaginary image picking up means installed at an arbitrary position. 

Parking assistance apparatus according to claim 2, characterized in that said picked up Image displayed on said 
screen has undergone viewpoint conversion processing to convert an Image picked up by said image picklng-up 
means to an image picked up by an imaginary image picking up means installed at an arbitrary position. 

Parking assistance apparatus according to claim 15. characterized In that said picked up Image displayed on 
said screen has undergone viewpoint conversion processing to convert an image picked up by said image picking- 
up means to an image picked up by an imaginary image picking up means Installed at an arbitrary position. 

Parking assistance apparatus according to claim 21 , characterized in that said picked up Image has undergone 
the viewpoint conversion processing where a straight line In the real space is represented in an image after view- 
point conversion. 

25. Parking assistance apparatus according to claim 22, characterized in tliat said picked up image has undergone 
the viewpoint conversion processing where a straight line in the real space Is represented In an Image after view- 
point conversion. 

26. Parking assistance apparatus according to claim 23, characterized in that said picked up Image has undergone 
the viewpoint conversion processing where a straight line in the real space Is represented in an Image after view- 
point conversion. 

27. Parking assistance apparatus according to claim 1, characterized in that said parking assistance apparatus dis- 
plays said guide lines In accordance with the rules of perspective. 

Parking assistance apparatus according to claim 2, cliaracterlzed in that said parking assistance apparatus dis- 
plays said guide lines In accordance with the rules of perspective. 

Parking assistance apparatus according to claim 15, characterized in that said parking assistance apparatus 
displays said guide lines in accordance with the rules of perspective. 

Parking assistance apparatus according to claim 3, characterized In that said parking assistance apparatus dis- 
plays, rather than said guide lines, points equidistantiy arranged by a predetermined distance on said guide lines. 

Parking assistance apparatus according to claim 9, characterized In that said parking assistance apparatus dis- 
plays, rather than said guide lines, points equidistantiy arranged by a predetermined distance on said guide lines. 

Parking assistance apparatus according to claim 16, characterized in that said parking assistance apparatus 
displays, rather than said guide lines, points equidistantiy arranged by a predetermined distance on said guide lines. 

Parking assistance apparatus according to claim 3, characterized in that said guide lines are displayed by chang- 
ing the display colors of the areas split by said guide lines. 

Parking assistance apparatus according to claim 9. characterized in that said guide lines arie displayed by chang- 



17 



BNSDOCID: <EP 1253065A2J_> 



EP 1 253 065 A2 



ing the display colors of the areas split by said guide lines. 

35. Parking assistance apparatus according to claim 15, characterized in that said guide lines are displayed by 
changing the display colors of the areas split by said guide lines. 

5 

36. Parking assistance apparatus that displays on a screen an Innage picked up by image picklng-up means mounted 
on a vehicle, characterized in that said parking assistance apparatus comprises control means including means 
for splitting said screen into a first area for displaying an area In the neighborhood of a vehicle and a second area 
for displaying the area except said area in the close proximity of the vehicle, means for viewpoint-converting an 

10 image of the neighborhood of a vehicle out of said image to an image where a straight line in the real space is 

represented in a straight line on the screen and displaying the resulting image in said first area, means for displaying 
in said second area an image of the neighborhood of a vehicle out of said image as a panoramically viewpoint- 
converted image continuous to the image displayed in said first area, and means for displaying the boundary of 
said first area and said second area. 

15 

37. Parking assistance apparatus according to claim 36, characterized in that said means for displaying the boundary 
displays separately the range where said vehicle can pass and the range where the vehicle cannot pass. 

38. Parking assistance apparatus according to claim 1 , further comprising 

^0 control means Including means for splitting said screen into a first area for displaying an area in the neigh- 

borhood of a vehicle and a second area for displaying the area except said area in the close proximity of the vehicle, 
means for viewpoint-converting an image of the neighborhood of a vehicle out of said image to an Image 
where a straight line in the real space is represented in a straight line on the screen and displaying the resulting 
image in said first area, 

means for displaying in said second area an image of the neighborhood of a vehicle out of said image as a 
panoramically viewpoint-converted image continuous to the image displayed In said first area, and 
means for displaying the boundary of said first area and said second area. 

39. Parking assistance apparatus according to claim 2, further comprising 

control means including means for splitting said screen Into a first area for displaying an area in the neigh- 
borhood of a vehicle and a second area for displaying the area except said area in the close proximity of the vehicle. 

means for viewpoint-converting an image of the neighborhood of a vehicle out of said image to an image 
where a straight line in the real space is represented in a straight line on the screen and displaying the resulting 
image in said first area, 

means for displaying in said second area an image of the neighborhood of a vehicle out of said Image as a 
panoramically viewpoint-converted image continuous to the image displayed in said first area, and 
means for displaying the boundary of said first area and said second area. 

40. Parking assistance apparatus according to claim 15, further comprising 

control means including means for splitting said screen into a first area for displaying an area in the neigh- 
borhood of a vehicle and a second area for displaying the area except said area in the close proximity of the vehicle, 

means for viewpoint-converting an image of the neighborhood of a vehicle out of said image to an image 
where a straight line in the real space is represented in a straight line on the screen and displaying the resulting 
image in said first area, 

means for displaying in said second area an Image ot the neighborhood of a vehicle out of said image as a 
panoramically viewpoint-converted image continuous to the Image displayed In said first area, and 
means for displaying the boundary of said first area and said second area. 

41. Parking assistance apparatus that displays on a screen an image shot by image picking-up means mounted on a 
so vehicle, characterized in that said parking assistance apparatus comprises control means for superimposing on 

said picked up image a guide showing the range where said vehicle passes when said vehicle is backed. 

42. Parking assistance apparatus according to claim 1 , further comprising control means for superimposing on said 
picked up Image a guide showing the range where said vehicle passes when said vehicle is backed. 



55 



43. Parking assistance apparatus according to claim2, further comprising control means for superimposing on said 
picked up image a guide showing the range where said vehicle passes when said vehicle is backed. 
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44. Parking assistance apparatus according to claim 15, further connprlsing control means for superimposing on said 
picked up image a guide showing the range where said vehicle passes when said vehicle is backed. 
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FIG.9A 
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